Look at Gridded NEXRAD Data

This is NEXRAD data from AWS (NOAA/Unidata) that has been gridded by Scott Collis using PyART

import xarray as xr
from distributed import Client, LocalCluster
import pyart
import matplotlib.pyplot as plt
import hvplot.xarray
import glob
import numpy as np

cluster = LocalCluster()
client = Client(cluster)
cluster

/Users/mgrover/opt/anaconda3/envs/tracer-dev/lib/python3.9/site-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 50357 instead
  warnings.warn(

LocalCluster

31d59f4d

Dashboard: http://127.0.0.1:50357/status	Workers: 4
Total threads: 12	Total memory: 16.00 GiB
Status: running	Using processes: True

Scheduler Info

Scheduler

Scheduler-6a1cd6ee-7195-4d53-8403-59dc80b4ce20

Comm: tcp://127.0.0.1:50358	Workers: 4
Dashboard: http://127.0.0.1:50357/status	Total threads: 12
Started: Just now	Total memory: 16.00 GiB

Workers

Worker: 0

Comm: tcp://127.0.0.1:50369	Total threads: 3
Dashboard: http://127.0.0.1:50375/status	Memory: 4.00 GiB
Nanny: tcp://127.0.0.1:50364
Local directory: /Users/mgrover/git_repos/tracer-radar-analysis/notebooks/march-22-2022/dask-worker-space/worker-ibm0o96k

Worker: 1

Comm: tcp://127.0.0.1:50370	Total threads: 3
Dashboard: http://127.0.0.1:50373/status	Memory: 4.00 GiB
Nanny: tcp://127.0.0.1:50362
Local directory: /Users/mgrover/git_repos/tracer-radar-analysis/notebooks/march-22-2022/dask-worker-space/worker-2gg3nsal

Worker: 2

Comm: tcp://127.0.0.1:50371	Total threads: 3
Dashboard: http://127.0.0.1:50374/status	Memory: 4.00 GiB
Nanny: tcp://127.0.0.1:50361
Local directory: /Users/mgrover/git_repos/tracer-radar-analysis/notebooks/march-22-2022/dask-worker-space/worker-4f3kav0n

Worker: 3

Comm: tcp://127.0.0.1:50372	Total threads: 3
Dashboard: http://127.0.0.1:50376/status	Memory: 4.00 GiB
Nanny: tcp://127.0.0.1:50363
Local directory: /Users/mgrover/git_repos/tracer-radar-analysis/notebooks/march-22-2022/dask-worker-space/worker-uyl2yy3x

Spin up a Cluster

ds = xr.open_mfdataset('../../data/nexrad-grid/xxx*')
ds_lat_lon = ds.set_index(x='lon',
                          y='lat')

ds_lat_lon

<xarray.Dataset>
Dimensions:       (time: 11, z: 31, y: 601, x: 601)
Coordinates:
  * time          (time) datetime64[ns] 2022-03-22T12:01:25.512000 ... 2022-0...
  * z             (z) float64 0.0 500.0 1e+03 ... 1.4e+04 1.45e+04 1.5e+04
  * y             (y) float64 26.74 26.75 26.76 26.77 ... 32.1 32.11 32.12 32.13
  * x             (x) float64 -98.1 -98.09 -98.08 ... -92.08 -92.07 -92.06
Data variables:
    reflectivity  (time, z, y, x) float32 dask.array<chunksize=(1, 31, 601, 601), meta=np.ndarray>
    rain_z        (time, z, y, x) float32 dask.array<chunksize=(1, 31, 601, 601), meta=np.ndarray>
    ROI           (time, z, y, x) float32 dask.array<chunksize=(1, 31, 601, 601), meta=np.ndarray>

xarray.Dataset

• Dimensions:
  • time: 11
  • z: 31
  • y: 601
  • x: 601
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2022-03-22T12:01:25.512000 ... 2...

    array(['2022-03-22T12:01:25.512000000', '2022-03-22T12:07:49.171000000',
           '2022-03-22T12:20:27.792000000', '2022-03-22T12:26:51.568000000',
           '2022-03-22T12:39:40.174000000', '2022-03-22T12:52:18.840000000',
           '2022-03-22T12:58:31.574000000', '2022-03-22T13:11:11.177000000',
           '2022-03-22T13:17:06.922000000', '2022-03-22T13:29:20.674000000',
           '2022-03-22T13:42:12.186000000'], dtype='datetime64[ns]')

  • z
    (z)
    float64
    0.0 500.0 ... 1.45e+04 1.5e+04

    standard_name :

    projection_z_coordinate

    long_name :

    Z distance on the projection plane from the origin

    axis :

    Z

    units :

    m

    positive :

    up

    array([    0.,   500.,  1000.,  1500.,  2000.,  2500.,  3000.,  3500.,  4000.,
            4500.,  5000.,  5500.,  6000.,  6500.,  7000.,  7500.,  8000.,  8500.,
            9000.,  9500., 10000., 10500., 11000., 11500., 12000., 12500., 13000.,
           13500., 14000., 14500., 15000.])

  • y
    (y)
    float64
    26.74 26.75 26.76 ... 32.12 32.13

    array([26.739926, 26.748913, 26.7579  , ..., 32.113977, 32.122963, 32.13195 ])

  • x
    (x)
    float64
    -98.1 -98.09 ... -92.07 -92.06

    array([-98.098944, -98.08888 , -98.078817, ..., -92.078654, -92.06859 ,
           -92.058527])

• Data variables: (3)
  • reflectivity
    (time, z, y, x)
    float32
    dask.array<chunksize=(1, 31, 601, 601), meta=np.ndarray>

    long_name :

    Reflectivity

    units :

    dBZ

    standard_name :

    equivalent_reflectivity_factor

    valid_max :

    94.5

    valid_min :

    -32.0

    Array Chunk Bytes 469.85 MiB 42.71 MiB Shape (11, 31, 601, 601) (1, 31, 601, 601) Count 33 Tasks 11 Chunks Type float32 numpy.ndarray

  • rain_z
    (time, z, y, x)
    float32
    dask.array<chunksize=(1, 31, 601, 601), meta=np.ndarray>

    long_name :

    rainfall_rate_from_z

    units :

    mm/h

    standard_name :

    rainfall_rate

    valid_max :

    500

    valid_min :

    0

    comment :

    a= 300.0 b=1.4

    Array Chunk Bytes 469.85 MiB 42.71 MiB Shape (11, 31, 601, 601) (1, 31, 601, 601) Count 33 Tasks 11 Chunks Type float32 numpy.ndarray

  • ROI
    (time, z, y, x)
    float32
    dask.array<chunksize=(1, 31, 601, 601), meta=np.ndarray>

    long_name :

    Radius of influence for mapping

    units :

    m

    standard_name :

    radius_of_influence

    Array Chunk Bytes 469.85 MiB 42.71 MiB Shape (11, 31, 601, 601) (1, 31, 601, 601) Count 33 Tasks 11 Chunks Type float32 numpy.ndarray

• Attributes: (0)

kazr_lat = 29.67
kazr_lon = -95.05899

ds_lat_lon.isel(z=10, time=0).reflectivity.plot(cmap='pyart_HomeyerRainbow',
                                                vmin=-10,
                                                vmax=60)
plt.scatter(kazr_lon, kazr_lat, marker='o', s=100, color='k')

<matplotlib.collections.PathCollection at 0x191490d90>

subset = ds_lat_lon.sel(y=kazr_lat,
                        x=kazr_lon,
                        method='nearest')

nearest_lat = np.round(subset.y.values, 4)
nearest_lon = np.round(subset.x.values, 4)

ref = subset.reflectivity.hvplot.quadmesh(x='time',
                                          label=f'Subset Nearest KAZR Radar \n Actual Lat:{kazr_lat} Lon:{kazr_lon} \n Nearest Lat: {nearest_lat} Lon: {nearest_lon}',
                                          clim=(-20, 60),
                                          cmap='pyart_HomeyerRainbow')

ref

/Users/mgrover/opt/anaconda3/envs/tracer-dev/lib/python3.9/site-packages/bokeh/core/property/bases.py:259: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
  return new == old
/Users/mgrover/opt/anaconda3/envs/tracer-dev/lib/python3.9/site-packages/bokeh/core/property/bases.py:259: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
  return new == old

start = ds.time.min()
end = ds.time.max()

ds.sel(x=0, y=range(20000, 40000, 5000)).reflectivity.plot(x='time',
                                                           col='y',
                                                           vmin=-20,
                                                           vmax=40,
                                                           cmap='pyart_HomeyerRainbow');

Look at KAZR

kazr_files = sorted(glob.glob('../../data/kazr/houkazrcfrgeM1.a1.20220315*'))

kazr_ds = xr.open_mfdataset(kazr_files,
                       use_cftime=True,
                       chunks={'time':2000})
kazr_ds['time'] = kazr_ds.indexes['time'].to_datetimeindex()

/Users/mgrover/opt/anaconda3/envs/tracer-dev/lib/python3.9/site-packages/xarray/coding/times.py:351: FutureWarning: Index.ravel returning ndarray is deprecated; in a future version this will return a view on self.
  sample = dates.ravel()[0]

from bokeh.models import DatetimeTickFormatter

def apply_formatter(plot, element):
    plot.handles['xaxis'].formatter = DatetimeTickFormatter(hours='%m/%d/%Y \n %H:%M',
                                                            minutes='%m/%d/%Y \n %H:%M',
                                                            hourmin='%m/%d/%Y \n %H:%M')
def plot_kazr(ds, max_height=8000):
    ds['time'] = ds.time.astype(int)/10**6
    
    ref = ds.reflectivity.compute()
    vel = ds.mean_doppler_velocity.compute()
    
    max_height = 9000
    ref_plot = ref.compute().hvplot.quadmesh(x='time', 
                                             y='range',
                                             rasterize=True,
                                             cmap='pyart_HomeyerRainbow',
                                             ylim=(0, max_height),
                                             height=300,
                                             width=700,
                                             clim=(-20, 40),
                                             label='KAZR Reflectivity (dBZ)',
                                             xlabel='Date (UTC)',
                                             hover=True).opts(hooks=[apply_formatter])

    vel_plot = vel.compute().hvplot.quadmesh(x='time', 
                                             y='range',
                                             rasterize=True,
                                             cmap='pyart_balance',
                                             ylim=(0, max_height),
                                             height=300,
                                             width=700,
                                             clim=(-8, 8),
                                             label='KAZR Doppler Velocity (m/s)',
                                             xlabel='Date (UTC)',
                                             hover=True).opts(hooks=[apply_formatter])
    
    return (ref_plot + vel_plot).cols(1)
    

plot_kazr(kazr_ds)